Papers with reference resolution accuracy
I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue (2025.findings-acl)
Copied to clipboard
| Challenge: | Using representational co-speech gestures, face-to-face interaction participants resolve references to objects using speech and gestures. |
| Approach: | They propose a multimodal reference resolution task centred on representational gestures . they propose 'self-supervised' pre-training approach to gesture representation learning that grounds body movements in spoken language. |
| Outcome: | The proposed approach aligns with expert annotations and has significant predictive power. |